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ABSTRACT 

Increasing knowledge integration has gained wide-spread support as an important goal in en¬ 
gineering education. The Chemical Engineering Pillars curriculum at the University of Pittsburgh, 
unique for its use of block scheduling, is one of the first four-year, integrated curricula in engineer¬ 
ing, and is specifically designed to facilitate knowledge integration. As this curriculum is one of 
the first of its kind, conventional assessment strategies are not necessarily well suited for evalua¬ 
tion. In fact, our assessment strategy - by necessity - includes two separate measurement goals. 
First, we focus on measuring the effectiveness of the new curricular structure for enhancing the 
ability of students to engage in systems thinking (knowledge integration). At the same time, we 
specifically assess the impact of this type of curriculum on students’ performance in conceptual¬ 
izing (chemical) engineering principles. The goal of this paper is to outline an overall assessment 
structure suitable for measuring the efficacy of multi-year integrated curricula. Specifically, we 
describe the currently available assessment vehicles under use as well as discuss the methods - 
and results - for building the additional assessment materials required for a thorough vetting of 
this type of (chemical engineering) curriculum. While our specific assessment is still ongoing, we 
include a portion of the results to date and discuss their implications for the portability of inte¬ 
grated curricula in chemical engineering. 

Keywords: integrated curricula, assessment, chemical engineering 
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INTRODUCTION 

The challenge of developing a better chemical engineering curriculum, or indeed any engineering 
curriculum, is to build it such that it prepares students for the engineering needs of today, while 
enabling them - through a strong and well integrated core of engineering knowledge - to maintain 
versatility through life-long learning and continuing education. Prevailing wisdom from engineering 
educators both within the US [1-3] and in the European Chemical Engineering Universities, Work¬ 
ing Party Education Group [4] is that the ideal engineering curriculum focuses on the following 
three issues: 

1. Giving the students a strong fundamental foundation by concentrating on the essential core of 
scientific and engineering basics, including biological applications and molecular insight [5, 6]. 

2. Enhancing systems thinking [7] by helping students to integrate their knowledge across courses 
and disciplines [8] so that they are better prepared to address open-ended problems. 

3. Preparing and providing for continuing education and life-long learning [4]. 

The strong focus throughout engineering on establishing broad-based systems thinking or 
knowledge integration within a discipline [1, 9,10] is not new; in fact, in the early to mid 1990’s the 
National Science Foundation funded a number of Coalitions that primarily championed the “inte¬ 
grated curriculum”. Until recently [11], however, there have been no educational efforts that have 
extended this (successful) approach beyond the Freshman [1, 10, 12] or Sophomore [13, 14] years 
of undergraduate engineering programs. At the University of Pittsburgh, we have developed an 
integrated curriculum (the Pillars curriculum) that spans the upper-class years for an undergradu¬ 
ate chemical engineer. Our fully integrated chemical engineering curriculum is unique for its use 
of block scheduling [15]—a technique with a strong literature base and proven track record in K-12 
education — for the first time in a traditional higher education engineering curriculum. Block sched¬ 
uling, in its simplest form, is transforming multi-semester courses into a single-semester course via 
extended, concentrated contact time. 

In accordance with the above-mentioned recommendations, the stated goals of the Pillars cur¬ 
riculum are to: (a) give students a strong fundamental foundation by concentrating on the essential 
core of scientific and engineering basics in a given discipline; and (b) enhance systems thinking by 
helping students to integrate core knowledge across traditional course boundaries so that they are 
better prepared to address open-ended problems. 

As this curriculum is one of the first of its kind, it is critical that it be evaluated not only for its 
effectiveness in enhancing the ability of students to engage in systems thinking, but also to spe¬ 
cifically assess the impact of this type of curriculum on students’ performance in conceptualizing 
(chemical) engineering principles. In particular, while the use of block-scheduling in K-12 education 
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has shown great promise for enhancing knowledge integration [16], it has been suggested that 
internalizing complex concepts like those in undergraduate engineering curricula requires repeti¬ 
tion [17]. In this paper, we outline a multi-faceted evaluation approach for an integrated (chemical 
engineering) curriculum with the specific aim of informing this seeming disparity. This includes 
not only articulating a plan for assessing both the degree of knowledge integration (via the use 
of concept maps) and the depth of the students’ conceptual knowledge (via the use of concept 
inventories), but also describing the details and results from the identification of target concepts 
for the development of new chemical engineering concept inventories, where necessary. This level 
of detail being devoted simply to an assessment plan is required because the general framework 
of the pillar classes is portable not only to other chemical engineering schools, but also to other 
engineering disciplines. Building a comprehensive body of data can then serve both as a validation 
of our own methods (and motivation for others to follow our model) and a road-map for evaluating 
the success of translation efforts. 


BACKGROUND - THE PILLARS CURRICULUM 

Current engineering instruction is often compartmentalized within a traditional 3-4 credit per 
course schedule, so that knowledge is disconnected and well-defined relationships are established 
across a curriculum only during the senior year, if at all [10]. By moving to a block-scheduled cur¬ 
riculum, we have integrated complementary subject-matter with experiments and open-ended 
problems, so that students see connections across the discipline during each course. While individual 
concepts within the discipline were redistributed for the purposes on enhancing integrated insight, 
the overall content covered in the curriculum remained largely unchanged. 

Logistically, the Pillars are a series of six high credit-count (5 or 6-credit) courses with comple¬ 
mentary 1-credit laboratories in the areas of Foundations (Mass/Energy Balances, simple Sepa¬ 
rations), Thermodynamics, Transport Phenomena, Reactive Processes (including more Complex 
Separations) and Process Systems Engineering I (Modeling/Control) and II (Design). Students 
typically are enrolled in one Pillar class each term for six consecutive terms - from sophomore 
through senior year. Students receive a single grade for each of these Pillar courses; however, the 
laboratory is graded separately each term. A brief description of each of the Pillars is included 
below. 

The Foundations of ChE pillar course combines elements of mass and energy balances, thermo¬ 
dynamics, separations, and product design. This course introduces chemical engineering problem 
solving techniques from both a (traditional) process-centric viewpoint as well as a product-centric 
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viewpoint. The course spans from theoretical (basic thermodynamics) to applied (separations) 
allowing a simple route to problem-based learning of difficult theoretical concepts. 

The Thermodynamics pillar course combines ideas from both pure and multi-component thermo¬ 
dynamics. It introduces molecular insight and the tools (including commercial software) for solving 
both simple and complex problems in phase and chemical equilibria. The course has a strong focus 
on multi-scale analysis, for example, covering intermolecular potentials (molecular-scale) to aid 
students in choosing equations of state for novel materials (macro-scale). 

The Transport Phenomena pillar course stresses analogies between momentum, mass, and heat 
transport. Content spans from the molecular origins of transport up through continuum descrip¬ 
tions, as well as macroscopic balances. 

The Reactive Processes pillar course integrates reactor design, reaction kinetics, and advanced 
separation processes to allow the comprehensive study of systems ranging from polymerization 
reactors to enzyme-catalyzed metabolism to bio-artificial organs. 

The Dynamics and Modeling class is the first of a two-part Systems Engineering pillar sequence. 
This course covers dynamical analysis of process systems, process control fundamentals, feedback, 
basic process modeling, and optimization. The second course in this sequence is the Design course 
which formally combines topics from all other pillars to allow both product and process design. 

As mentioned, to properly assess this novel curricular structure we need to test not only for the, 
presumably positive, impact of block scheduling on knowledge integration, but ascertain whether 
the lack of content repetition has a negative impact on conceptual learning. To accomplish these 
two goals we employ Concept Mapping, to measure knowledge integration, and apply (and develop, 
as necessary) Concept Inventories, to measure conceptual understanding. 


CONCEPT MAPS - KNOWLEDGE INTEGRATION 


Concept maps are graphical representations of a student/subject’s thoughts, theories, and/or con¬ 
cepts and their relative organization [18]. In practice, when developing a concept map, the student/ 
subject draws a diagram showing a hierarchy of ideas or concepts linked through branches between 
the sub-concepts, with further links showing interrelationships between inter-branch ideas/concepts, 
when necessary (i.e., cross-links). A critical feature of a concept map is that it includes not only a hier¬ 
archy of ideas linked and cross-linked, but that those linkages are labeled in such a way as to clearly 
articulate the meaning of those relationships. A schematic of a concept map is shown in Figure 1. 

Concept mapping was initially devised as a technique for measuring the assimilation of scientific 
knowledge in children [19, 20] and has subsequently found a variety of uses in pedagogy including 
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Figure 1. Schematic of a Concept Map. Shown is an example concept map displaying links, 
branches and cross-links. 


teaching, learning, planning, and assessment [21]. In the context of its use in the current work, concept 
maps have been shown to be a valuable assessment tool for evaluating the extent of knowledge integra¬ 
tion exhibited by a student/subject [18, 22, 23]. This observation makes concept mapping an attractive 
tool for evaluating integrated curricula, and they are used in that capacity in the present work. 


Scoring Concept Maps 

While there has been some question over the years as to the validity of various methods of inter- 



1 

2 
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Comprehensiveness - cov¬ 
ering completely, broadly 

The map lacks subject definition; the 
knowledge is very simply and/or limited. 
Limited breadth of concepts (i.e., minimal 
coverage of coursework, little or no men¬ 
tion of employment and/or lifelong learn¬ 
ing). The map barely covers some of the 
qualities of the subject area. 

The map has adequate subject 
definition, but knowledge is 
limited in some areas (i.e., much 
of the coursework is mentioned, 
but one of two of the main aspects 
are missing). Map suggests a 
somewhat narrow understanding 
of the subject matter. 

The map completely defines the 
subject area. The content lacks 
no more than one extension area 
(i.e., most of the relevant 
extension areas, including 
lifelong learning, employment, 
people, etc. are mentioned). 

Organization - to arrange 
by systematic planning and 
united effort 

The map is arranged with concepts only 
linearly connected. There are few (or no) 
connections within/between the branches. 
Concepts are not well integrated. 

The map has adequate organization 
with some within/between branch 
connections. Some, but not complete, 
integration of branches is apparent. A 
few feedback loops may exist. 

The map is well organized with 
concept integration and the use 
of feedback loops. Sophisticated 
branch structure and connectivity. 

Correctness - conforming 
to or agreeing with fact, 
logic, or known truth 

The map is naive and contains misconcep¬ 
tions about the subject area; inappropri¬ 
ate words or terms are used. The map 
documents an inaccurate understanding of 
certain subject matter. 

The map has few subject matter in¬ 
accuracies; most links are correct. 
There may be a few spelling and 
grammatical errors. 

The map integrates concepts 
properly and reflects an accurate 
understanding of subject matter 
meaning little or no misconcep¬ 
tions, spelling/grammatical errors. 


Figure 2. Concept Map Scoring Rubric 
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Figure 3. Two Example Concept Maps, (top) Shown is a map that lacks detail(focusing 
only on 1 course within ChE), is very linear in nature, and includes some mistakes in the 
linkages, (bottom) Shown is a map that is considerably more complete (but that still lacks 
societalenvironmental, and industrial ChE context), has some cross-links between related 
branches of the map, and contains no glaring errors 
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preting or “scoring” concept maps [24, 25] and the relationship of these scores to overall student 
achievement, it is of interest here that these maps can be evaluated both based on the included 
content as well as on the structure of the map itself [21]. In this way, there is a built-in control whereby 
the content-related score(s) can be compared to the content-related assessments (outlined in the 
next section). All of our maps are scored using a rubric [18] which gives independent scores for the 
elements of comprehension, organization, and correctness (see Figure 2). The mapping exercise can 
be at the end of each academic year or, alternatively, at the “capstone” or graduation stage. In our 
project, we have performed this exercise for both scheduling methods, traditional and integrated 
curriculum. In this way we can assess the students’ “knowledge integration” not only as a function 
of curriculum followed, but also temporally within the Pillars curriculum. 


Preliminary Concept Map Results 

To ensure anonymity of the participants and unbiased scoring of the maps, it takes multiple years 
of data collection before comprehensive results are available. Therefore, a detailed analysis of the 
impact of our model curriculum on “Knowledge Integration” will be the subject of a future publica¬ 
tion. Nevertheless, included here are example maps from two students (Figure 3) and a summary of 
our concept map scores to date, reported in Figures 4 and 5. Note that, in scoring these maps, we 
distinguish between three cohorts of student - traditional (no Pillar courses), transitional (students 
have taken one or more Pillar courses), and Pillar cohorts - and further subdivide the Pillar cohort 


Comprehensiveness Organization Correctness 


Total 


I Traditional 
| Pillar 


° 2 


GO 


iJ ii ii ll 


Figure 4. Shown are the mean concept map scores for all maps that have been scored 
to date. Note that the three shades of blue denote Pillars students that have completed 
sophomore, junior, or seniors years (light to dark), respectively 
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Figure 5. Shown are the concept map score distributions for all maps that have been scored to 
date. Note that the three shades of blue denote Pillars students that have completed sophomore, 
junior, or seniors years (light to dark), respectively. Also, arrows at top denote the median of 
each distribution. Finally, note that the light blue (Pillar sophomores) scores are remarkably 
similar to those of the red (traditional seniors), when comparing the combined (Total) scores 


into results from sophomores and juniors as well as seniors. When comparing the scores obtained, 
there is an encouraging trend that not only do Pillars students (seniors: 2.92 “Total” median score) 
have a higher score on each measure versus the traditional cohort group (seniors: median 2.42), but 
they are achieving comparable scores to the traditional cohorts earlier in their careers (sophomores: 
median 2.42; juniors: median 2.75). Perhaps more encouraging is to compare the distribution of 
scores (Figure 5). From Figure 5 we see that there is almost no difference between the distribution 
of the traditional (senior) cohort and the Pillars’ Sophomore cohort. Thus, we are encouraged that 
our goal of enhancing knowledge integration early in our students’ studies is being achieved. 

Concept Inventories - Conceptual Understanding 

While the traditional exams used in engineering courses are well suited to gauging a student’s 
problem-solving skills, they are often not aimed at measuring conceptual understanding [26]. 
Moreover, the variability in test questions from term-to-term and instructor-to-instructor make 
it difficult to use student grades to compare learning from inherent differences across semesters 
even within the same course [27]. 

Concept inventories (CIs) [26-29] are standardized tests that are specifically designed to evalu¬ 
ate students’ conceptual understanding and thus are ideal as a tool for the evaluation of a student’s 
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fundamental mastery of the conceptual material within a curriculum. In addition to serving as a valid 
assessment of conceptual knowledge on its own [27], the fact that CIs focus on misconceptions - 
that is, incorrect world-views that block a student’s ability to to make connections between basic 
concepts [26] - this method of assessment is particularly well suited to evaluating a curriculum 
whose goal is improving knowledge integration. 

The Pillars curriculum consists of six Pillar courses - Foundations, Thermodynamics, Transport 
Phenomena, Reactive Processes, and a two-part Process Systems Engineering sequence - for which 
only two existing CIs are applicable. This section describes the few relevant CIs available as well 
as our efforts to construct the remaining required inventories. It is useful to note that, given the 
fundamental structure of the pillars curriculum, specific application areas of each of the pillar sub¬ 
ject areas are purposely left fluid. In this way, concept inventories can be effectively built because 
the student achievement that is being assessed is their understanding/attainment of fundamental 
engineering principles, rather than the use of engineering knowledge in a specific application area. 
Thus, many of the Cl questions should be portable within the discipline. 


Existing Concept Inventories 

A concept inventory study at the Colorado School of Mines (CSM) [28, 29] developed a Cl fo¬ 
cused on the Thermal and Transport Sciences. This well-written and extensively validated Cl [28] has 
been in use over several years at a number of institutions. By splitting this Cl, as suggested by the 
developers [29], into parts corresponding to Thermodynamics, Fluid Mechanics, and Heat Transfer, 


You are in the business of melting ice at 0 C using hot blocks of metal as an energy source. One option 
is to use one metal block at a temperature of 200 C and a second option is to use two metal blocks each 
at a temperature of 100 C. 

All the metal blocks are made from the same material and have the same weight and surface area. Which 
option will melt more ice? 

a. the 100 C blocks 

b. the 200 C block 

c. either option will melt the same amount of ice 

d. can’t tell from the information given because: 

e. 2 blocks have twice as much surface area as 1 block so the energy transfer rate will be higher 
when more blocks are used 

f. energy transferred is proportional to the mass of blocks used and the change in block 
temperature during the process 

g. using a higher temperature block will melt the ice faster because the larger temperature difference 
will increase the rate of energy transfer 

h. the temperature of the hotter block will decrease faster as energy is transferred to the ice 

i. the heat capacity of the metal is a function of temperature 


Figure 6. Concept Inventory Example Question - CSM test [28, 31, 32]. Used by 
permission 
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questions from this Cl are appropriate for use with the Thermodynamics and Transport Phenomena 
(using the Fluid Mechanics and Heat Transfer questions) Pillar courses, respectively. 

The CSM concept inventory [30] was developed using a rigorous “Delphi study” [30] (discussed 
in detail in the next section) and went through an extended beta testing period [28]. The Cl ques¬ 
tions derived using this procedure, an example of which is shown in Figure 6, were impressive 
enough that we modeled our Cl development procedure after that followed at CSM. Of particular 
importance to note is how the question couples a test of the concept with the rationale behind the 
answers as an internal consistency check of student effort. Also, note that “reasonable” distractors 
are used based on the identified symptoms and rationale behind the misconception. 

New Concept Inventories - Delphi Study 

As mentioned previously, the Pillars curriculum aims to produce improvements in student un¬ 
derstanding of major ChE concepts [33], which not only include Thermodynamics and Transport 
Phenomena, but also Reaction Kinetics (RK), Process Design and Control (PS) and Material and 
Energy Balances (plus Separations, MES). While the CSM concept inventory is aptly suited to the 
Thermodynamics and Transport Pillars, CIs for the remaining subject areas are yet to be developed. 
This section details the procedure - called the Delphi method [30] - that we have followed to collect 
the common misconceptions that should be tested in concept inventories aimed at these subjects. 

The purpose of a Delphi study is to gather and condense the judgments of experts about a par¬ 
ticular subject under conditions of anonymity, and therefore without bias [34]. In a Delphi study, 
the ideas generated by the participants are independent, isolated and anonymous. Thus, the impact 
of choice-shift (/.e., the polarization of opinions due to strong personalities or name recognition) is 
minimized while the opinions are being collected. 

In a conventional Delphi Study method, there is no communication between the participants. 
Rather, information flow is orchestrated by a director and occurs through feedback questionnaires 


Dear Respondent: 

Thank you for agreeing to participate in our Delphi study of difficult concepts in chemical engineering. 
As we previously indicated, the study will be conducted in four rounds. In this round, we would like 
you to list concepts in material and energy balances, reactive processes (kinetics, reactors, and advanced 
separations), and process systems that you think are either important or difficult for students. Please submit 
as many items as you would like in each of the categories. 

We would like to have this round completed by MM/DD/YYYY so that we can promptly consolidate your 
responses and move on to the next round. 

Figure 7. Delphi Study Round 0 Correspondence - Concept Gathering 
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Dear Respondent: 

Thank you for your participation in our Delphi study. As a reminder, the purpose is to identify and rank 
difficult concepts for undergraduate students in chemical engineering. Based on your Round 0 responses, we 
have consolidated the important and difficult concepts. Your task now is to rank the consolidated concepts for 
understanding (U) and importance (I) on a scale from 0-10, where: 

0 - No one understands OR it is not important to understand this concept 
10 - Everyone understands OR it is extremely important to understand this concept 

You will not have to rank any concept for which you do not feel you have sufficient expertise. You may “skip” 
the concepts that you do not feel comfortable rating. Please complete your ratings by MM/DD/YYYY. 

Figure 8. Delphi Study Round 1 Correspondence - Concept Rating and Refinement 


that are periodically sent to the participants. By structuring the group communication in this manner, 
the process is considered effective at allowing a group of experts to identify the specific charac¬ 
teristics of a problem, and ways to better deal with the problem [34]. In our case, the problem at 
hand is the identification of commonly misunderstood yet fundamental concepts relevant to each 
of the 3 Pillar subject areas under consideration (RK, PS and MES). 

For our study, 15 external participants were chosen, representing nationally renowned chemical 
engineering faculty from highly ranked programs. The opinion-gathering was coordinated by three 
local subject area “champions” and was done in a series of rounds to first gather and then refine 
the data. The Round 0 correspondence requested participation in the study and stimulated inter¬ 
est by offering a small monetary incentive. At this stage, the participants were asked to identify 
a list of commonly misunderstood concepts which were of fundamental importance to any core 
undergraduate ChE curriculum. The particular invitation is included in Figure 7. 

Round 0 yielded independent concepts in each of the subject areas as follows, 44 RK, 33 MES, 
34 PS and 11 general or cross-cutting concepts (GEN). The responses to this round were refined by 
the local subject area champions in two ways. Concepts that could be coalesced were combined, 
while those that had been identified by only a single participant were eliminated. 

For Round 1, the condensed questionnaire results (consisting of 36 RK, 26 MES, 26 PS, and 11 
general or cross-cutting concepts) were electronically sent to the respondents in order to ascer¬ 
tain their opinion on the group responses. We specifically requested that they express the degree 
to which they agree or disagree that a particular concept was commonly misunderstood and of 
high importance on a scale of 0-10. This second invitation is shown in Figure 8. This initial ranking/ 
scoring of the concepts for each subject area can be seen schematically in Figure 9, based on the 
mean score of the group. 
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Figure 9. Round 1 responses. The dashed line represents the proposed cutoff 
for responses that are high in importance and high in misunderstanding (or low in 
understanding). The numbers correspond to a particular misconception 


Round 2 was aimed at accounting for variation in absolute score values from participant-to- 
participant. As such, the participants were again asked to rate each concept both for “understand¬ 
ing” and “importance”, but were now given the median score as well as the 50% range of the group 
for each concept. They were then asked to revise their responses based on the consensus scores 
of the group. They were specifically instructed to justify any of their scores that continued to fall 
outside of the 50% range. The third invitation is included in Figure 10 and the resultant spread of 
concept scores are shown in Figure 11. 
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Dear Respondent: 

Thank you for your participation in our Delphi study. As a reminder, the purpose is to identify and rank 
difficult concepts for undergraduate students in chemical engineering. Based on your Round 0 responses, we 
have consolidated the important and difficult concepts. Your task now is to rank the consolidated concepts for 
understanding (U) and importance (I) on a scale from 0-10, where: 

0 - No one understands OR it is not important to understand this concept 

10 - Everyone understands OR it is extremely important to understand this concept 

You will not have to rank any concept for which you do not feel you have sufficient expertise. You may “skip” 
the concepts that you do not feel comfortable rating. Please complete your ratings by MM/DD/YYYY. 


Figure 10. Delphi Study Round 2 Correspondence - Further Concept Refinement 


Comparing Figure 11 to Figure 9 shows the shift in importance and/or understanding of some con¬ 
cepts when the participants had access to the opinions of others within the group. These responses 
were further refined by defining a cut-off for how high in importance a concept could be while also 
being poorly misunderstood. Figure 11 shows how the cutoff line was used to cull the list of concepts 
at this stage of the process (the same line is shown on Figure 9 for illustrative purposes, it was not 
used at that stage). Implementing the cutoff line after the Round 2 results were obtained resulted in 
18 RK, 13 MES, 15 PS and 8 general or cross-cutting (GEN) concepts that could be considered high 
in both importance and probability of misconception (i.e., conceptual difficult). 

The progressive refinement of high-importance and hard-to-understand concepts as a function 
of Delphi round is shown in Table 1. By Round 3 (comprising only of our local champions), the final 
list was condensed to 5 RK, 6 MES, 6 PS concepts along with 2 GEN misconceptions covering all 
3 Pillar areas. It should be noted that this final list was built from a local champion’s translation of 
the expert participants’ responses, and the reductions were confirmed in a final, post-Delphi, cor¬ 
respondence. The reason that this step represented a rather dramatic reduction in the number of 
“concepts” is that the experts’ responses in the previous rounds often represented difficult topics 
or the symptoms of underlying misconceptions rather than actual concepts that gave students dif¬ 
ficulty. In this post-Delphi round, the champions, therefore, combined many of these student issues 
under the heading of more general misconceptions. 

Student misconceptions 

The misconceptions that were identified in each of the subject areas are outlined here, including 
the underlying student misconception as well as the “symptoms” that lead to each of the misconcep¬ 
tions. As with the CSM concept inventory, questions derived from this list will be aimed at probing 
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Figure 11. Round 2 responses. Dashed line represents the proposed cutoff for responses 
which were high in importance and high in misunderstanding (or low in understanding) 


the misconception and will include distractors inspired by the listed symptoms. Also, in analogy to 
the work at CSM, we will pilot the questions with a number of different focus groups to tease out 
biases within the wording or question construction. 

The general or cross-cutting misconceptions include: 

1 Misconception: All questions are answerable and have one “right” answer. 

• Students do not realize that assumptions are necessary in order to evaluate data. 

• Students do not realize that assumptions and measurement variability will lead to open- 
ended problems/solutions. 
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Round 

RK 

MES 

PS 

Reduction Mechanism 

0 

44 

33 

34 

Initial Responses 

1 

36 

26 

26 

Consolidation by champions 

2 

18 

13 

15 

Cutoff line implemented 

3 

5 

6 

6 

Refinement by champions 


Table 1. This illustrates the refinement of the number of truly fundamental yet 
misunderstood concepts with each successive round. 


• Students do not realize that a “degree of freedom” analysis is necessary to evaluate whether 
enough information is even available to make a solution possible. 

2 Misconception: Math and core sciences do not directly impact “engineering problems”. 

• Students do not know where to start in tackling open-ended problems. 

• Students have difficulty translating ChE problem solving from a “petrochemicals” viewpoint 
to other fields (biological applications, energy, etc.). 

The misconceptions related to the Material and Energy Balances (with Separation, MES) 
subject area include: 

MES -1. Misconception: all/no processes run at unsteady state. 

• Students confuse recycle/bypass with unsteady processes. 

• Students do not know how to write unsteady balances. 

MES -2. Misconception: flow work only applies to “piston problems”. 

• Students do not understand when flow work must be considered (open v. closed). 

MES -3. Misconception: state variables are similar to process variables. 

• Students do not understand the intrinsic path-independence of state functions. 

• Students do not understand the need for reference states. 

MES- 4. Misconception: explicitly calculating the degrees of freedom is unnecessary. 

• Students are unclear on (in)dependence of balance equations. 

• Students are unclear on (in)dependence of reactions. 

• Students do not understand how thermodynamics (phase behavior) impacts the degrees 
of freedom of a system. 

• Students are not clear on how/when to choose appropriate control volumes for performing 
balances. 

MES- 5. Misconception: The ideal gas law is always appropriate (even for liquids/solids). 
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MES- 6. Misconception: Significant figures are unimportant, two decimal places is about 
right. 

The misconceptions identified in the Reactive Processes (RK) area include: 

RK -1. Misconception: Reaction kinetics and rate expressions must be known a priori rather than 
derived or measured. 

• Students have difficulty determining the relationship between reaction rate expressions/ 
laws and molecular, stochastic and chemical kinetics. 

• Students struggle to extract intrinsic kinetics from reactor modeling. 

RK -2. Misconception: Transport phenomena impacts reactor behavior in mass-transfer limited 
cases. 

• Students have trouble predicting/analyzing residence time distributions. 

• Students cannot reconcile the behavior of non-ideal and multiphase reactors (e.g. bubble 
columns, fluidized, packed, fixed and entrained beds) with the idealized cases they can 
analytically describe. 

• Students have little intuition regarding the appropriateness of correlations given accept¬ 
able error. 

• Identifying the proper experimental/theoretical tests for heat and mass transfer limitations 
in reactors is often difficult. 

RK -3. Misconception: Mass balances are only useful for overall material accounting due to the 
presence of reactions. 

• Students struggle to translate and reconcile molecular and atomic or elemental species bal¬ 
ances on reactive systems (including selectivity, yield, and extent of reaction concepts). 

• Students can often not identify/determine linear independence of chemical reactions, rate 
determining step, elementary and non-elementary reactions, and complex reaction path¬ 
ways. 

• The physical meaning of operating and equilibrium lines is elusive. 

RK - 4. Misconception: Biological system applications (e.g. metabolic engineering, controlled 
drug delivery, bio-separations) are analyzed in a completely different manner than purely 
“chemical” applications. 

RK - 5. Misconception: Reactor stability and safety are the same thing. 

The misconceptions identified in the Process Systems (PS) area include: 

PS -1. Misconception: Optimization of processes and flow-sheets is straightforward. 

• Students have difficulty defining, differentiating, and formulating objectives and constraints 
(includes incorporation of environmental, sustainable, and efficiency issues within objec¬ 
tives and constraints). 
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• Students have difficulty converting word problems with data to a formal mathematical 
(optimization) statement (true of continuous, batch, and synthesis problems). 

• Students have only a limited introduction to optimization as undergraduates; topic is often 
covered only late in the degree. 

• Students do not understand changes in solution required to handle multivariable optimiza¬ 
tion (search vectors) vs. 1-dimensional problems. 

PS -2. Misconception: All details are equally important all the time; there is no hierarchy in de¬ 
sign. 

• Students do not understand how to simplify process descriptions without losing process 
characteristics. 

• Students cannot identify when to employ rigorous (formal) optimization vs. heuristic deci¬ 
sions. 

• Students do not understand the impact of economics on decision-making (magnitude and 
timing in design procedure). 

PS -3. Misconception: With suitable simplification, processes can be treated as single-phase and 

ideal. 

• Multi-phase processes (heterogeneous catalysis, packed/fluidized beds) and spatially 
inhomogeneous processes (bubble columns) are underutilized in class due to complexity, 
leading to students not grasping the fundamental challenges of these systems. 

• PS -4. Misconception: Designers lay out plants, control engineers make and keep them 
running. 

• Students do not connect network principles (reactor-separation-recycle or “pinch” systems) 
with control challenges. 

• Impact of process dynamics on (in)feasibility, especially for highly integrated systems, is 
poorly grasped. 

• Students do not differentiate concepts of “design” and “synthesis”. 

PS -5. Misconception: Time-domain models are difficult to construct and do not provide in¬ 
sight. 

• Linear algebra and ODE skills are weak due to lack of use (time since taking the courses, 
if taken at all). 

• State space modeling is introduced only in the dynamics and control class (in general). 

• Dynamical analysis concepts (time constants, gains, zero and delay dynamics) and synthe¬ 
sis concepts (closed-loop rates of response, effect of parameter tuning) are not taught in 
the time domain (students lack the math skills/motivation to explore these effects without 
suitable motivation (homework, exam) in a course). 
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PS -6. Misconception: Models and/or processes generate data, and data is “truth” that cannot 

be changed. 

• Students cannot identify root cause of output from process data (troubleshooting and 
need for/benefit from retrofitting). 

• Students do not grasp impact of (or correction for) change in process scale on dynamics 
or feasibility. 

• Students cannot identify process data characteristics that suggest the need for and benefit 
from particular advanced control strategies. 

Having rigorously identified the important misconceptions in each of the three subject areas that 
are unrepresented by CIs, the next step is to develop and test candidate questions targeting these 
concepts. Using questions geared toward the aforementioned concepts in each of the Cl focus areas, 
we will employ the strategy used by Miller eta/. [28] whereby focus groups of students will provide 
feedback via “thinking aloud” interviews after attempting the candidate questions to aid in question 
refinement. The results of these exercises will be the subject of a future communication. 


SUMMARY AND CONCLUSIONS 

Assessment is without doubt a critical component to the successful development and growth of 
pedagogical innovation. The difficulty in developing a truly novel curriculum is that the strategies 
used to assess a traditional curriculum are not necessarily well suited to its evaluation. The Pillars 
curriculum at the University of Pittsburgh is perhaps the first fully integrated engineering curriculum. 
Through the coordinated use of both concept mapping - in order to evaluate the degree of knowledge 
integration - and concept inventories - to evaluate the students’ conceptual understanding - we 
have developed a strategy tailored specifically for the evaluation of integrated engineering curricula. 
A unique difficulty of our approach is that assessment vehicles and techniques needed to be devel¬ 
oped in parallel with the development, implementation, and initial assessment of the curriculum. In 
particular, over the course of our project we have identified the misconceptions and begun to build 
concept inventories in the subject areas of Reaction Kinetics, Process Design and Control and Mate¬ 
rial and Energy Balances (plus Separations). Through this paper, we expect that others considering 
integrated engineering curricula may have a road map for how to build a comprehensive assessment 
strategy. At the same time, we hope that chemical engineers will find the misconception database of 
use, while other engineering disciplines can find utility in the methodology used to identify same. 
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